Week 8: Expectation Maximization

So far, we discussed clustering algorithms that involve a hard assignment of each datapoint to a cluster, typically based on its proximity to other points in that cluster. However, simply assigning points to the nearest cluster is not always adequate to capture more complex structure. For example, the lecture slides show an example where one cluster is much larger and less dense than another. We’ll discuss a probabilistic clustering method that can capture this by introducing two new ingredients: soft assignment of points xi to each cluster, and using additional parameters to discuss the shape of a cluster, so as to capture the fact that clusters can be large, small, sparse, or dense.